Add configurable HTTP connection eviction, expose ConnectionId APIs#130476
Conversation
|
Tagging subscribers to this area: @karelz, @dotnet/ncl |
There was a problem hiding this comment.
Pull request overview
This PR adds new experimental connection-level diagnostics and control points to System.Net.Http’s SocketsHttpHandler, including a configurable pooled-connection eviction callback and a connection identifier that’s propagated through connection establishment, request sending, and plaintext stream filtering. It also adds functional coverage across HTTP/1.1, HTTP/2, and HTTP/3 (including Alt-Svc).
Changes:
- Introduces experimental public APIs:
SocketsHttpHandler.ShouldEvictConnection,SocketsHttpConnectionEvictionContext, andHttpRequestMessage.ConnectionId(plusConnectionIdsurfacing in connect/plaintext-filter contexts). - Implements eviction evaluation in the connection pool and stamps
ConnectionIdonto requests for HTTP/1.1, HTTP/2, and HTTP/3. - Adds functional tests (mostly OuterLoop) exercising eviction behavior and connection id propagation, plus warning suppressions for the experimental diagnostic ID.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/libraries/System.Net.Http/tests/UnitTests/System.Net.Http.Unit.Tests.csproj | Links Experimentals.cs into unit tests to support experimental attribute usage/constants. |
| src/libraries/System.Net.Http/tests/FunctionalTests/System.Net.Http.Functional.Tests.csproj | Suppresses SYSLIB5008 for functional tests consuming experimental APIs. |
| src/libraries/System.Net.Http/tests/FunctionalTests/SocketsHttpHandlerTest.cs | Adds extensive functional coverage for eviction callback behavior and ConnectionId stamping across protocols and scenarios. |
| src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.AltSvc.cs | Adds HTTP/3 Alt-Svc tests validating eviction context endpoint reporting. |
| src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/SocketsHttpPlaintextStreamFilterContext.cs | Adds ConnectionId to plaintext stream filter context (experimental). |
| src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/SocketsHttpHandler.cs | Adds ShouldEvictConnection experimental callback API. |
| src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/SocketsHttpConnectionEvictionContext.cs | New public experimental context type for eviction decisions. |
| src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/SocketsHttpConnectionContext.cs | Adds ConnectionId to connect callback context (experimental). |
| src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionSettings.cs | Plumbs eviction callback through handler settings cloning/normalization. |
| src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPoolManager.cs | Adjusts pool maintenance timer cadence when eviction callback is configured. |
| src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionBase.cs | Adds eviction state/context and evaluation plumbing at the connection level. |
| src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs | Stamps HttpRequestMessage.ConnectionId on HTTP/1.1 requests. |
| src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/Http3Connection.cs | Ensures HTTP/3 requests get ConnectionId and eviction context reports the connected endpoint. |
| src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/Http2Connection.cs | Stamps HttpRequestMessage.ConnectionId on HTTP/2 requests. |
| src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/ConnectionPool/HttpConnectionPool.Http3.cs | Passes connected endpoint into HTTP/3 connection establishment for eviction context accuracy. |
| src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/ConnectionPool/HttpConnectionPool.Http2.cs | Threads connection id through HTTP/2 connection construction/filtering paths. |
| src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/ConnectionPool/HttpConnectionPool.Http1.cs | Threads connection id through HTTP/1.1 connection construction/filtering and triggers eviction evaluation on return. |
| src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/ConnectionPool/HttpConnectionPool.cs | Adds eviction evaluation pass scheduling and clears ConnectionId across retry attempts; allocates connection id before connect for correlation. |
| src/libraries/System.Net.Http/src/System/Net/Http/HttpRequestMessage.cs | Adds experimental ConnectionId property to requests. |
| src/libraries/System.Net.Http/src/System/Net/Http/BrowserHttpHandler/SocketsHttpHandler.cs | Adds ShouldEvictConnection stub throwing PNSE for browser handler. |
| src/libraries/System.Net.Http/src/System.Net.Http.csproj | Suppresses SYSLIB5008 for internal use and includes new source file + Experimentals.cs. |
| src/libraries/System.Net.Http/ref/System.Net.Http.cs | Adds the new public API surface to the ref assembly contract. |
| src/libraries/Common/src/System/Experimentals.cs | Defines the experimental diagnostic ID constant for the new APIs. |
| docs/project/list-of-diagnostics.md | Registers the new experimental diagnostic ID in the central list. |
|
PTAL @dotnet/ncl. I'll improve the tests, but the product side of the change should be ready for review |
rzikm
left a comment
There was a problem hiding this comment.
Impl. LGTM, didn't look at tests too much
|
Workflow state for the Holistic Review Orchestrator. {
"version": 5,
"last_dispatched_commit": "2dc0efb170dcddb8585627706134cae225392fd4",
"last_dispatched_base_ref": "main",
"last_dispatched_base_sha": "1e589a260a7b94b4a646f50a53d06766548eae26",
"last_reviewed_commit": "2dc0efb170dcddb8585627706134cae225392fd4",
"last_reviewed_base_ref": "main",
"last_reviewed_base_sha": "1e589a260a7b94b4a646f50a53d06766548eae26",
"last_recorded_worker_run_id": "29766958360",
"review_attempt_commit": "",
"review_attempt_base_ref": "",
"review_attempt_count": 0,
"max_review_attempts": 5,
"review_history_format": "holistic-review-disclosure-v1",
"review_history": [
{
"commit": "8be3be0246abdbddae29313435dedb293a184761",
"review_id": 4730675022
},
{
"commit": "2dc0efb170dcddb8585627706134cae225392fd4",
"review_id": 4737835331
}
]
} |
There was a problem hiding this comment.
Holistic Review
Motivation: Well justified. Both new APIs (SocketsHttpHandler.ShouldEvictConnection + SocketsHttpConnectionEvictionContext, and HttpRequestMessage.ConnectionId) are backed by approved proposals (#130102, #130108, both carrying api-approved), and the scenario—evicting healthy pooled connections in response to DNS/address changes without recycling purely via PooledConnectionLifetime—is real and previously required ceremony over EventSource telemetry.
Approach: Reasonable and well-integrated. A single connection id is allocated up front in ConnectAsync and threaded through ConnectCallback, plaintext filter, telemetry, the eviction context, and the request message, giving one consistent correlation id. Eviction runs off the maintenance timer outside the pool lock with snapshots, a generation counter re-evaluates HTTP/1.1 connections missed while busy, and callback exceptions are swallowed so a misbehaving callback can't take down pool maintenance. The XML docs are unusually thorough, especially around the CONNECT-tunnel id distinction. Both approved API shapes match the ref exactly.
Summary: HttpRequestMessage flag refactor drops the atomic Interlocked.CompareExchange that enforced single-send and newly mutates _flags mid-request (via the ConnectionId setter from each connection's SendAsync), introducing potential races with the non-atomic Disposed/AlreadySent writes. A networking maintainer should confirm whether that relaxation is intentional. Note the PR is marked NO-MERGE pending API review completion, and one maintainer (rzikm) has already approved.
Detailed Findings
⚠️ Thread-safety — HttpRequestMessage flag refactor drops atomicity (see inline comment)
The conversion of the packed int _sendStatus (mutated atomically via Interlocked.CompareExchange/|=) to a plain MessageFlags _flags field removes the atomic claim in MarkAsSent and adds a new mid-request writer through the ConnectionId setter. Previously _flags-equivalent writes after send were effectively single-threaded; now request.ConnectionId = Id runs inside connection SendAsync and can overlap a concurrent Dispose() (_flags |= Disposed) or a racing send, tearing a non-atomic read-modify-write. Detailed inline comment on HttpRequestMessage.cs. Worth a maintainer's explicit confirmation that single-send detection and flag integrity are still guaranteed.
✅ API conformance — matches approved proposals
The ref/ surface (ShouldEvictConnection, SocketsHttpConnectionEvictionContext with Age/ConnectionId/HttpVersion/DnsEndPoint/RemoteEndPoint, and ConnectionId on the connect/plaintext/request contexts) matches the shapes approved in #130102/#130108, including the [Experimental("SYSLIB5008")] diagnostic and nullable IPEndPoint? RemoteEndPoint. list-of-diagnostics.md and Experimentals.cs are updated consistently.
✅ Concurrency of eviction evaluation — reasonable guards
Callback invocation is gated by _evictionEvaluationInProgress (pool-level) and _evictionCallbackInProgress (per-connection) plus a generation counter; snapshots for HTTP/2 and HTTP/3 are taken under SyncObj and the callback is awaited outside the lock. _connectionDisposalCts is canceled (not disposed) on close so in-flight callback tokens stay valid. The Interlocked.Exchange(ref _evictionEvaluationInProgress, true) / _evictionCallbackInProgress uses rely on the generic Interlocked.Exchange<T> over bool, which is valid. The author explicitly notes the benign race of running the callback more than once per generation.
✅ Eviction cadence and lifetime interaction
When a callback is set, the maintenance timer is capped to fire at least every 5s regardless of a large/infinite idle timeout, giving eviction a predictable cadence; CheckExpirationOnGet/OnReturn/IsUsable all newly short-circuit on MarkedForEviction so a marked connection is retired via get/return/scavenge without serving new requests. _markedForEviction is volatile, appropriate for the cross-thread read.
💡 Test coverage — good breadth, timer-dependent
Tests exercise Alt-Svc/HTTP/3 authority reporting, origin-endpoint reporting, and ConnectionId correlation across versions; several are [OuterLoop] because they wait on the pool maintenance timer, which is appropriate. Consider (follow-up, non-blocking) a targeted test asserting single-send / ConnectionId-stamping behavior under concurrent sends to lock in the _flags semantics discussed above.
Note
This review was generated by this repository's Holistic Review agentic workflow to complement the built-in Copilot review.
Generated by Holistic Review · 247.4 AIC · ⌖ 11.4 AIC · ⊞ 10K
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 25 out of 25 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (3)
src/libraries/System.Net.Http/src/System/Net/Http/HttpRequestMessage.cs:236
- HttpRequestMessage.MarkAsSent is no longer atomic (it used to use Interlocked.CompareExchange). This can allow multiple concurrent sends of the same request instance to slip through CheckRequestMessage without throwing, and it also introduces Enum.HasFlag boxing on this hot path. Consider using an atomic flag set on the underlying int representation and switch the subsequent HasFlag checks to bitwise ops.
internal bool MarkAsSent()
{
MessageFlags previousFlags = _flags;
_flags = previousFlags | MessageFlags.AlreadySent;
return !previousFlags.HasFlag(MessageFlags.AlreadySent);
}
src/libraries/System.Net.Http/src/System/Net/Http/HttpRequestMessage.cs:163
- ConnectionId getter uses Enum.HasFlag, which boxes and can allocate. This property is set/reset in the SocketsHttpHandler send loop, so it’s worth keeping allocation-free by using a bitwise flag check.
get => _flags.HasFlag(MessageFlags.ConnectionIdSet) ? _connectionId : null;
src/libraries/System.Net.Http/src/System/Net/Http/HttpRequestMessage.cs:250
- Disposed getter uses Enum.HasFlag, which boxes and can allocate. Since CheckDisposed is used by multiple setters, using a bitwise check avoids avoidable allocations on a frequently-hit path.
get => _flags.HasFlag(MessageFlags.Disposed);
There was a problem hiding this comment.
Holistic Review
Motivation: Unchanged and well justified. Both new APIs (SocketsHttpHandler.ShouldEvictConnection + SocketsHttpConnectionEvictionContext, and HttpRequestMessage.ConnectionId) are backed by approved proposals (#130102, #130108), enabling eviction of healthy pooled connections in response to DNS/address changes without relying solely on PooledConnectionLifetime.
Approach: Refined since the last review. The eviction evaluation loop in HttpConnectionPool.EvaluateConnectionsForEviction was changed from an async method that awaited each connection's callback one-by-one to a synchronous method that starts each per-connection check as fire-and-forget (_ = connection.EvaluateForEvictionAsync()). This removes the previous pool-level _evictionEvaluationInProgress guard, which is safe: the maintenance timer (HttpConnectionPoolManager.RemoveStalePools) is single-threaded per manager (a one-shot Timer.Change restarted only at the end of each pass), so passes don't overlap; per-connection overlap is still guarded by _evictionCallbackInProgress via Interlocked.Exchange, and EvaluateForEvictionAsync catches all exceptions internally so the discarded task never faults. The change correctly prevents a single slow/blocking user callback from stalling evaluation of the other pooled connections (and subsequent passes) — a real improvement over the prior serial-await design. The remaining approach (single up-front connection id threaded through connect/plaintext/telemetry/eviction/request, generation-counter re-evaluation of busy HTTP/1.1 connections, callback run outside the pool lock) is unchanged.
Summary: HttpRequestMessage flag refactor (packed int _sendStatus → plain MessageFlags _flags) drops the atomic single-send claim and adds a mid-request _flags writer via the ConnectionId setter, a relaxation a networking maintainer should explicitly confirm. That code lives in earlier commits unchanged by this update, so it is carried forward in this cumulative assessment rather than re-raised as a new finding. The PR also remains marked NO-MERGE pending API-review completion.
Assessment History
- review 4730675022 (commit
8be3be0246ab): verdict was⚠️ Needs Human Review; current verdict is⚠️ Needs Human Review. Changed in reasoning while the verdict is unchanged: the initial review flagged the eviction concurrency as "reasonable guards" but noted the pool-level guard; commits 12–14 replaced the serial-await design (and its pool-level_evictionEvaluationInProgressguard) with a fire-and-forget design that better tolerates slow callbacks, so the approach assessment improved. The verdict itself is held solely by the unchangedHttpRequestMessageflag-atomicity concern and the NO-MERGE/API-review status, neither of which these commits touch.
Detailed Findings
No new actionable findings in the incremental changes (commits a4176bb9ca27, eb1e44388f3a, 2dc0efb170dc). The fire-and-forget eviction refactor is correct: the maintenance pass is single-threaded, per-connection overlap remains guarded, and the awaited callback body swallows exceptions so discarding the task is safe. Test changes are limited to expanding/restructuring existing eviction and ConnectionId coverage and read correctly. The previously noted HttpRequestMessage flag-atomicity concern is in code unchanged by these commits and is not re-raised here per incremental-review scope.
Note
This review was generated by this repository's Holistic Review agentic workflow to complement the built-in Copilot review.
Generated by Holistic Review · 119.2 AIC · ⌖ 11.8 AIC · ⊞ 10K
|
/azp run runtime-libraries-coreclr outerloop |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
|
Thanks @rzikm Not seeing any http outerloop failures, so marging this now. I might follow up with minor tweaks later |
Closes #130102
Closes #130108
Implements the new APIs from the two issues above (NO-MERGE while those are still going through API review).
More details around the intended behavior are in those issues.